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-available  to  a reasoner  was  limited  by  virtue  of  being  incomplete. 
The  incompleteness  in  the  information  arose  because,  as  is 
typical  of  most  real-life  situations,  only  a sample  of  a 1 1 
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possible  locjical  expressions  that  could  be  formulated  from  a 
number  of  independent  dimensions  were  assigned  truth  values. 
Empirical  studies  with  college  students  were  carried  out.  with  a 
previously  formulated  concept-attainment  model  that  allowed  for 
multiple  concept  solutions.  Using  both  a concept-production  and 
a concept-recognition  format  several  factors  influencing  success 
in  concept  attainment  were  isolated. 


A novel  logical  model  for  instance  deduction,  the  obtaining  of 
valid  instances  from  concepts,  was  formulated.  The  model 
complemented  the  previous  concept-attainment  model.  Extensive 
empirical  studies  were  carried  out  based  on  the  instance  deduction 
model.  A case-study  approach  to  several  sessions  of  performing  J 
instance  deductions  was  undertaken  as  well  as  production  and 
recognition  experiments  with  groups  of  observers.  The  conclusion 
was  reached  that  the  instance-deduction  treatment  described  by  the 
model  — which  was  in  accord  with  acknowledged  canons  of  formal 
logic  --  could  easily  be  acquired  by  untrained  participants  and  lead 
to  quick  improvement  in  making  practical  deductions.  Theoretical 
analysis  concentrated  or.  two  comparisons  in  the  logical  structure 
of  problems:  (l)  between  conjunctive  and  disjunctive  concepts;  JiJ 

(2)  between  conjunctive  and  disjunctive  instances.  Only  the 
second  distinction  was  found  to  be  important,  v Factors  were 
described  that  relate  to  the  difficulties  individuals  experienced 
in  dealing  with  disjunctive  instances.  \ 
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TO:  Chief  of  Naval  Research,  Arlington,  Virginia  22217 

Attention:  Engineering  Psychology  Programs,  Code  455 

Drs.  M.  A.  Tolcott  and  J.  J.  O'Hare 

FROM:  Dr.  Bruce  M.  Ross,  Center  for  Research  in  Thinking  and 

Language,  Department  of  Psychology 

The  Catholic  University  of  America,  Washington,  D.C.  20064 

SUBJECT:  Final  report  of  work  completed  under  the  support  of 

Contract  N00014-7 5-C-0933 , Work  Unit  No.  NR  197-025, 
between  the  Catholic  University  of  America  and  th^ 
Engineering  Psychology  Programs,  Office  of  Naval  Research. 

I.  This  report  constitutes  a final  description  of  work  completed 
under  the  support  of  Contract  No.  N00014-75-C-09B3,  Work  Unit 
Number  NR  197-025,  between  the  Catholic  University  of  America 
and  the  Engineering  Psychology  Programs,  Office  of  Naval 
Research.  The  contract  was  initiated  1 May  1973  and  (with  a 
nine-months,  no-cost  extension)  terminated  31  January  1977. 

II.  The  aim  of  the  contract  was  to  study  deductive  thinking  under 
conditions  of  limited  information  from  both  a theoretical  and 
empirical  standpoint.  The  theoretical  perspective  resulted 
in  logical  models  of  deduction  that  were  used  in  designing  and 
analyzing  problems  administered  to  observers  in  the  empirical 
phase  of  the  research.  Four  technical  reports,  listed  at  the 
end  of  the  present  report,  were  completed  and  distributed. 

III.  The  present  report  is  divided  into  five  sections.  Three  short 
sections  (A,  B,  and  D)  are  concerned  with  the  rationale  and 
justification  of  the  research.  Two  longer  sections  (C  and  E) 
describe  the  empirical  work  carried  out  under  the  contract. 
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A . General  Rationale  for  Studying  Deductive  Problem  Solving 

Individuals  often  have  information  available  to  them  from  which 
they  should  be  able  to  draw  straightforward  conclusions  with 
logical  certainty.  Yet  they  frenuently  fail  to  do  so,  even  though 
memory  is  not  involved  and  valid  implications  are  placed  in  a list 
among  other  solution  alternatives.  In  our  research  these 
considerations  led  to  an  attempt  to  answer  three  general  Questions 
(a)  What  kind  of  biases  prevent  implications  from  being  evaluated 
correctly?  (b)  What  types  of  problem  solving  strategies  are 
commonly  invoked?  (c)  To  what  extent  can  the  use  of  three  or 
four  simple  rules  for  deduction  facilitate  correct  deductions? 

To  avoid  misunderstanding,  it  should  be  emphasized  that  deductive 
reasoning  in  the  form  that  we  have  studied  it  is  usually  at  a 
simpler  level  than  that  undertaken  with  probabilistic  decision 
making.  Indeed,  some  deductions  are  often  necessary  before 
probability  calculations  are  worth  performing.  Further,  in  many 
situations  it  is  desirable  to  know  the  qualitative  possibilities 
that  are  logically  feasible  with  regard  to  whether  a quantitative 
probability  can  be  attached  to  each  individual  possibility.  That 
is  to  say,  drawing  deductions  sometimes  yields  a first  approxi- 
mation in  describing  problem  solutions  where  only  partial 
information  is  available.  With  strong  time  pressure  this  approxi- 
mation can  be  in  itself  sometimes  sufficient  to  encourage  or 
eliminate  a course  of  action. 

A practical  example  of  drawing  deductive  conclusions  in  a 
nautical  setting  was  furnished  by  describing  the  use  of  deduction 
in  target  surveillance  based  on  multiple  sightings.  in  such  an 
example  the  deductive  basis  for  drawing  conclusions  is  not  always 
limited  to  cases  where  physical  objects  exist,  since  negative 
examples  (nonexistent  configurations)  can  also  play  a role  in 
deductive  inferences. 

B . Advantages  of  the  Deductive  Models  used  in  the  Experiments 

In  the  first  experiments  work  was  based  on  a previously  formulated 
logical  model  for  presenting  concept-attainment  problems  (Mott 
& Ross,  1963) . By  concept-attainment  is  meant  the  deduction  of 
logically  valid  concepts  from  sets  of  positive  and ^or  neqative 
instances.  in  work  performed  under  the  contract  a model  was  also 
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developed  for  concept  instantiation  (instance  deduction)  problems 
(Mott  & Ross,  1975).  instance  deduction  refers  to  the  identifi- 
cation of  valid  instances  of  a set  of  one  or  more  concepts. 

This  instance  deduction  model  was  applied  to  the  later  experiments 
of  the  contract.  (Gurney  & Ross,  1977;  Ross  & Gurney,  1975). 

In  general,  both  of  the  above  logical  models  incorporated  three 
major  advantages  for  the  design  and  analysis  of  deductive  problem 
solving  experiments  over  previous  paradigms  described  in  the 

psychological  literature.  > 

These  advantages  are; 

(1)  Multiple  correct  solutions,  often  of  different  logical  types, 
were  correct  within  the  same  problem. 

(2)  Both  concepts  and  instances  (and  therefore  problem  solutions) 
could  be  either  coniunctive  or  disjunctive  in  fomi  depending 
on  the  problem  design  the  experimenter  chose. 

(3)  Valid  solutions  could  be  drawn  from  incomplete  instantiations. 

This  generalization  meant  that,  in  accord  with  most  real  life 
situations,  only  a sample  of  all  possible  instances  (concepts) 
needed  to  be  specified  as  positive  or  negative,  and  yet 
logically  consistent  solutions  could  be  determined. 

C . Empirical  Results  for  Concept  Attainment 

1.  Concept-Attainment  Production  Experiment 

Observers  (N=24)  were  presented  problems  (sets  of  instances) 
that  each  allowed  two  or  more  correct  concepts  to  be 
attained.  Minimal  instructions  were  given  to  present  possible 
solution  types  rather  than  to  illustrate  deductive  rules  to 
follow.  two  conditions  were  administered,  a Specified  condition 
where  the  number  of  correct  solutions  that  could  be  obtained 
for  each  problem  was  printed  and  an  Unspecified  condition  where 
no  solution  numbers  were  printed.  (Note  that  participants  in 
this  experiment  and  all  others  to  be  reported  were  male  and 
female  college  students  with  the  exception  of  the  addition  of  a 
few  university  staff  members  in  one  experiment.)  Four 
conclusions  were  reached.  One,  the  Specified  group  produced 
many  more  correct  and  incorrect  solutions  than  the  Unspecified 
group.  This  was  especially  true  for  the  production  of  correct 
lst-order  conjunctive  concepts  (A  and  B) . Second,  there  was 
a marked  reluctance  on  the  part  of  individuals  in  the 
Unspecified  group  to  produce  more  than  two  concepts  per  problem 
regardless  of  the  number  of  correct  concepts.  Third,  both 
Specified  and  Unspecified  groups  produced  few  lst-order 
disjunctions  (A  0£  B) , but  they  produced  far  too  many  2nd-order 
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disjunctions  (AB  or  C;  AB  or  CD)  that  were  needlessly  complex. 
This  frequent  listing  of  nonessential  values  among  attained 
concepts  suggested  that  observers  often  confused  concept 
instances  with  the  concepts  themselves.  Fourth,  individual 
performance  range  was  very  wide;  from  2 to  19  correct  concepts 
were  obtained  out  of  a possible  24. 

(Technical  Report  CRTL-TR-1) 

2.  Concept-Attainment  Recognition  Experiment 

instructions  were  more  detailed  than  for  the  previous 
experiment.  Four  worked-out  practice  problems,  with  four 
concepts  for  each  problem,  were  included  in  the  instructions. 

A total  of  20  correct  and  22  incorrect  concepts  were  distrib- 
uted over  10  concept-attainment  problems.  Observers  (N=40) 
encircled  Yes  or  No,  giving  a 50%  chance  level  of  correctness. 

An  approximate  BOy',  or  better  level  of  correctness  was 
obtained  for  0-order  concepts  (A  alone),  lst-order  conjunctive 
concepts  (both  with  2 and  with  3 values) , and  1st  order 
disjunctions  (both  with  2 and  with  3 values).  Again  participants 
did  poorly  with  2nd-order  disjunctions  recognizing  correctly 
only  58%  of  these  concepts.  it  was  concluded  that  naive 
observers  performing  with  a recognition  format  can  assimilate 
logical  instructions  sufficiently  to  attain  concepts  with  a 
single  logical  operator  at  a satisfactory  level  of  proficiency. 

(Technical  Report  CRTL-TR-1) 

D . Specific  Rationale  for  Studying  Instance  Deduction 

The  other  studies  performed  under  the  contract  were  devoted  to 
theoretical  and  empirical  investigations  of  instance  deduction. 

This  area  has  received  little  previous  attention  in  the  psycho- 
logical literature.  Usually  instances  have  been  defined  in  such 
a narrow  range,  most  often  as  2-dimensional  conjunctive  concepts, 
that  to  ask  for  instance  deduction  would  be  trivial.  With  the 
present  deductive  system,  however,  permitting  both  disjunctive  and 
conjunctive  instances  of  both  the  1st-  and  2nd-order,  the 
deduction  and  identification  of  positive  and  negative  instances 
can  be  a challenging  problem. 
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Nevertheless,  there  are  two  ways  in  which  instance  deduction  is  a 
simpler,  more  direct  process  than  concept  attainment.  This  being 
the  case,  studying  instance  deduction  can  often  be  a better 
vehicle  for  understanding  participants'  reasoning  than  concept  attain- 
ment. At  the  same  time  the  study  of  instance  deduction  is  of 
interest  in  its  own  right  since  individuals  can  hardly  be  consid- 
ered to  understand  a concept  if  they  can't  clearly  identify 
positive  and  negative  instances  of  the  concept.  The  first  way  in 
which  instance  deduction  is  mo>e  direct  is  that  each  concept  is 
considered  individually  as  possessing  multiple  instances.  Thus 
deduction  is  on  a single-step  basis  even  when  instances  must  be 
valid  for  several  concepts.  In  contrast,  concept  attainment  is 
more  complex  in  that  (except  for  redundant  instances)  all  instances 
are  interrelated  with  a modification  in  implied  concepts  for  each 
new  instance  that  is  added. 

The  second  way  in  which  the  instance  deduction  process  is  simpler 
than  concept  attainment  is  that  negative  instances  do  not  have  to 
be  considered  simultaneously  with  positive  instances,  nor  is 
all-negative  instantiation  a feasible  procedure  as  it  is  with 
concept  attainment.  Negative  instances  for  instance  deduction 
are  just  the  remaining  instances  that  are  not  valid  for  any  concepts 
after  determination  of  whether  such  instances  can  be  positive  for 
these  concepts.  Since  there  is  little  flexibility  in  procedure 
when  performing  instance  deduction,  deductive  processes  are  more 
straightforward  and  possibly  easier  to  understand  than  they  are 
for  concept  attainment  where  a variety  of  approaches  is  possible 
in  solving  the  same  problem. 

(Technical  Report  CRTL-TR-3  presents  an  account  and  justification 
of  instance  deduction  in  formal-logic  terminology.) 


1.  Instance-Deduction  Production  Experiment 

Ten  problems  that  listed  sets  of  concepts  were  presented  to 
observers  (N=30)  who  were  required  to  supply  two  instances 
per  problem.  Whether  instances  were  to  be  coniunctive  or 
disjunctive  was  specified;  siv  of  the  problems  specified  a 
mix  of  one  conjunctive  and  one  disjunctive  instance.  Four 
matching  rules  were  supplied  to  the  participants  relating  to 
the  four  possible  combinations  of  conjunctive  and  disjunctive 
instances  and  concepts.  The  number  of  correct  instances 
given  per  individual  ranged  from  2 for  two  observers  to  a 
perfect  20  for  three  observers.  The  two  individuals  who  scored 
only  2 would  have  produced  8 correct  instances  if  a minor  error 
of  incompletely  specifying  all  dimensions  was  overlooked. 
However  scored,  it  was  evident  that  this  experiment  like  the 
concept-attainment  production  experiment  resulted  in  a very 
large  range  of  scores  making  the  drawing  of  general 
conclusions  from  production  experiments  more  difficult  than 
recognition  experiments. 

Mean  accuracy  reached  was  quite  high,  as  might  be  expected 
since  each  set  of  concepts  had  many  more  than  two  correct 
instances.  Notwithstanding,  a significant  difference  was 
found  between  correct  production  of  conjunctive  instances  (3 4%) 
and  disjunctive  instances  (71%).  Greater  mean  accuracy  was 
also  shown  for  production  of  coniunctive  instances  on  a per 
problem  basis  when  both  conjunctive  and  disjunctive  instances 
were  requested  in  the  same  problem.  On  the  other  hand 
disjunctive  concepts  did  not  appear  to  be  more  difficult  to 
deduce  instances  from  than  were  conjunctive  concepts. 

These  results  suggested  that  it  was  not  difficulty  in  under- 
standing disjunctive  concepts  per  se  that  caused  problem 
difficulty  in  the  expanded  instance-deduction  experiment  to 
be  described  next  or  in  the  following  instance-recognition 
experiments.  More  exact  comparisons  based  on  this  theme  were 
made  at  a descriptive  level.  Two  factors  were  especially 
noted  as  producing  errors  in  instance  deduction.  One  was  the 


obvious  tendency  for  errors  to  increase  as  number  of  concepts 


increased.  More  interesting  was  the  finding  that  heterogenity 
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of  concept  levels  in  the  same  list  (a  mixture  of  O-order, 
lst-order,  and  2nd-order  concepts)  tended  to  cause  more  error 
than  when  all  concepts  were  at  the  same  level  (eg.,  all 
lst-order  concepts) . 

(Technical  Report  CRTL-TR-1) 

The  total  N for  the  above  experiment  was  increased  to  54  by 
adding  22  more  college  students  as  observers.  Three  of  the  10 
instance-production  problems  were  of  interest  in  order  to 
make  a clear  comparison  between  difficulty  of  conjunctions 
and  disjunctions.  In  a between-problem  comparison  (3  disjunc- 
tive concepts  given  and  2 conjunctive  instances  reauested 
versus  a second  problem  with  3 conjunctive  concepts  given  and 
2 disjunctive  instances  requested)  significantly  fewer  errors 
were  made  with  conjunctive  instances.  The  same  result  was 
obtained  with  an  intraproblem  comparison  where  both 
disjunctive  and  conjunctive  concepts  were  listed  and  both 
types  of  instances  were  requested.  Another  point  to  emphasize 
is  that  the  more  accurate  production  of  conjunctive  instances 
went  counter  to  probability  expectations  since  there  were 
almost  always  many  more  valid  disjunctive  than  conjunctive 
instances.  It  was  concluded  that  with  these  rather  tightly 
controlled  comparisons  it  was  not  the  misunderstanding  of 
disjunction  as  such  but  the  specific  inability  of  the 
subjects  to  formulate  disjunctions  as  instances  of  concepts 
that  increased  errors. 

(Technical  Report  CRTL-TR- 2 ) 

2.  Instance-Deduction  Recognition  Experiment 

This  experiment  was  carried  out  on  a larger  scale  than  the 
others  with  a total  N of  155  observers.  There  were  10  problems 
with  a total  of  42  instance  recognitions.  Participants  were 
divided  into  three  groups.  A Double-Scan  Rule  group  (N-53) 
had  available  the  four  rules  used  in  the  instance-production 
experiment.  A Double-Scan  Naive  group  (N=64)  and  a Single-Scan 
Naive  group  (,N=38  had  no  deduction  rules  available  to  them. 

The  difference  between  the  double-scan  and  single-scan 
procedures  was  that  with  the  double-scan  procedure  participants 
made  two  passes  through  the  problems. 
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On  the  first  pass  observers  indicated  whether  an  instance  was 
or  was  not  a positive  instance  of  all  the  listed  concepts; 
on  the  second  pass  they  indicated  whether  the  remaining  concepts 
were  instances  of  some  or  none  of  the  concepts  (although  they 
could  also  change  their  minds  about  their  original  judgments). 
With  the  single  scan  procedure  all,  some,  or  none  judgments 
were  made  on  the  initial  reading  of  a problem. 

There  was  a significantly  better  performance  for  the  rule 
conditions  with  30.9%  error,  as  compared  to  39.0%  error  for 
the  Double-Scan  Naive  group  and  38.4%  error  for  the  Single-Scan 
Naive  group.  Interesting  differences  were  found  in  response 
usage.  Rule-using  observers  tended  to  overuse  the  affirmative 
“all"  response  while  naive  observers  underused  it.  For  the 
naive  groups,  the  single-scan  group  overused  "some"  and  the 
double-scan  group  the  "none"  response.  Thus  to  some  extent  a 
psychological  set  appeared  to  be  induced  that  was  only  partly 
dependent  on  performance  level.  At  the  same  time,  though 
favoring  the  yes  response,  the  rule  group  had  errors  fairly 
evenly  distributed  over  the  three  response  categories,  an 
outcome  that  mirrored  correct  response  proportions.  The  finding, 
noted  above  in  the  instance-production  experiment,  of  much 
easier  identification  of  conjunctive  instances  from  disjunc- 
tive concepts  as  compared  to  the  converse  identification  of 
disjunctive  instances  from  conjunctive  concepts  was  highly 
supported  for  the  recognition  results. 

The  real  force  of  the  comparison  1 tween  disjunctive  and 
conjunctive  instances  in  the  production  and  recognition 
experiments  was  that  the  matching  rules  given  to  observers  to 
make  both  comparisons  were  almost  identical;  the  only  difference 
was  that  there  was  a reversal  of  concept  and  instance  as  to 
which  was  disjunctive  and  which  conjunctive.  (In  the  four 
rules  given  in  the  instructions  the  rules  involved  were  Nos. 

2 and  3.)  Taken  altogether,  the  results  suggested  that  it  was 
the  use  of  disjunctions  as  solution  outcomes  that  made  for 
performance  difficulty  rather  than  failure  to  comprehend  the 
meaning  of  disjunction.  A descriptive  model  was  outlined  in 
which  subjects  first  treated  all  instances  as  conjunctions  and 
subsequently,  if  required,  derived  disjunctions  from  these 
conjunctions.  In  this  model,  it  is  the  fact  that  obtaining 
disjunctive  instances  is  frequently  a two-step  process  that 
increases  errors. 

(Technical  Report  CRTL-TR-2) 
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3.  intensive  Analysis  of  Observers'  Performance  of  Instance- 
Deduction  Recognition 

Five  major  contrasts  can  be  drawn  between  this  experiment  and 
the  previously  reported  experiments.  One,  problems  were 
presented  at  the  observer's  convenience  on  a CRT-scope  display 
and  answers  were  automat ica lly  recorded . This  is  in  contrast 
to  the  paper-and-penci 1 booklet  format  of  previous  experiments. 
However,  printed  matching  rules  were  again  available  as  they 
had  been  in  the  prior  experiments.  Two,  correctness  feedback 
was  given  automatically  following  every  instance  recognition 
(all,  some,  or  none  responses)  while  no  feedback  was 
administered  in  earlier  experiments.  Three,  problems  were 
administered  to  the  same  individuals  for  five  sessions  (actually 
there  were  eight  sessions  in  all  but  three  early  sessions 
administering  easier  problems  were  omitted  from  analysis) 
rather  than  the  single-session  administration  of  th<^  other 
experiments.  Four,  reaction-time  measures  from  time  of  display 
to  time  of  responding  were  taken,  whereas  non^  had  been  taken 
previously.  Five,  all  listed  concepts  for  which  instances 
were  to  be  identified  were  complex  in  that  they  were  all 
either  2nd-order  conjunctions  or  2nd-order  disjunctions. 

Concepts  for  a particular  problem  were  all-conjunctive, 
all-disjunctive,  or  mixed.  Previous  experiments  had  only  a 
few  problems  consisting  exclusively  of  2nd-order  concepts. 

The  present  experiment  had  13  problems  with  the  same  10 
instances  listed  for  each  problem,  making  a total  of  130 
instance  recognitions  for  each  session.  The  price  that  was 
paid  for  the  large  amount  of  data  generated  by  each  observer 
was  to  restrict  the  number  of  participants  to  three,  which  put 
analysis  on  a case-study  basis  rather  than  that  of  statis- 
tical averages  as  had  been  true  for  the  earlier  experiments. 

Two  diverse  styles  of  problem  solving  were  delineated  by 
observer  performance.  As  sessions  progressed,  two  of  the 
participants  appeared  to  take  an  "abstract"  approach  leading 
to  large  step  increases  in  accuracy,  particularly  in  correctly 
identifying  disjunctive  instances.  The  other  observer  pursued 
a more  "concrete"  rote  learning  approach  with  gradual, 
though  significant  accuracy  increases.  Further,  this 
individual  did  not  show  any  especially  large  improvement  with 
disjunctive  instances.  The  "abstract"  observers  terminated 
with  an  overall  success  rate  of  near  90 % while  the  "concrete" 
observer  achieved  68%  correctness  in  the  last  session.  Thus 
some  success  was  achieved  even  with  an  inferior  approach. 
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The  results  obtained  from  error  tallies  were  paralleled  by 
reaction-time  measurements  compiled  separately  for  each 
session  and  problem  type.  The  "concrete"  observer  who 
responded  faster  than  the  "abstract"  observers  only  during 
the  first  session  also  showed  less  differentiation  in 
responding  to  disjunctive  and  conjunctive  instances  through- 
out all  sessions.  For  all  participants  there  was  a steady 
decrease  in  reaction  times  from  the  first  to  third  sessions 
followed  by  quite  level  mean  reaction  times  for  the  remaining 
two  sessions.  Reaction  times  did  not  completely  reflect 
observer  performance  since  correctness  continued  to  improve 
in  sessions  4 and  5 for  one  "abstract"  observer  and  the 
"concrete"  observer.  The  other  "abstract"  observer  had 
obtained  near  perfect  correctness  by  session  3. 

In  support  of  the  notion  that  concepts,  in  contrast  to 
instances,  are  at  about  the  same  level  of  difficulty  whether 
disjunctive  or  conjunctive,  no  reaction-time  differences  were 
found  between  all-disjunctive  and  all-conjunctive  concept 
problems.  As  compared  to  the  previously  obtained  results 
achieved  on  a one-session  basis  (CRTL-TR- 2 ) , a successful 
demonstration  was  given,  that  the  described  system  of  instance 
deduction  whose  only  pedagogical  feature  was  knowledge  of 
results  significantly  reduced  errors  for  untrained  observers 
dealing  with  moderately  complex  concepts.  Therefore  a main 
conclusion  was  that  given  matching  instructions,  exposure  to 
problems,  and  correctness  feedback  considerable  instance- 
deduction  improvement  can  occur  without  the  need  for  any 
individualized  instruction. 


(Technical  Report  CRTL-TR- 4 ) 
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